Optimizing Tutorial Planning in Educational Games: A Modular Reinforcement Learning Approach

Recent years have seen a growing interest in educational games, which integrate the engaging features of digital games with the personalized learning functionalities of intelligent tutoring systems. A key challenge in creating educational games, particularly those supported with interactive narrative, is devising narrativecentered tutorial planners, which dynamically adapt gameplay events to individual students to enhance learning. Reinforcement learning (RL) techniques show considerable promise for creating tutorial planners from corpora of student log data. In this paper, we investigate a modular reinforcement learning framework for tutorial planning in narrative-centered educational games, with a focus on exploring multiple modular structures for modeling tutorial planning sub-tasks. We utilize offline policy evaluation methods to investigate the quality of alternate tutorial planner representations for a narrativecentered educational game for middle school science, CRYSTAL ISLAND. Results show significant improvements in policy quality from adopting a data-driven optimized modular structure compared to a traditional monolithic MDP model structure, particularly when training data is limited.